Video Clustering

We address the issue of clustering of video images. We assume that video clips have been segmented into shots which are further represented by a set of keyframes. Video clustering is thus reduced to a clustering of still keyframe images. Experiments with 8 human subjects reveal that humans tend to use semantic meanings while grouping a set of images. A complete-link dendrogram constructed from the similarities provided by the subjects revealed two significant categories of images; that of city scenes and landscapes. A hierarchical clustering based on moments of 17 DCT coefficients of the JPEG compressed keyframe images reveals that ad hoc low-level features are not capable of identifying semantically meaningful categories in an image database. It is well known that a clustering scheme will always find clusters in a data set! In order to define categories that will aid in indexing and browsing of video data, features specific to a given semantic class should be used. As an example, we present initial results using multiple 2-class classifications. Our experiments have been conducted on two databases of 98 and 171 images, respectively. Classifiers for city/non-city shots, presence/absence of text in images, and presence of specific image textures (grass and sky) are being developed.